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San Onofre's Steam Generator Failures 
Could Have Been Prevented 



Summary 

Southern California Edison's four replacement steam generators at their San Onofre Nuclear 
Generating Station failed in less than two years of operation, while the original equipment 
operated for 28 years. Fairewinds has been analyzing the data in order to determine how such an 
expensive investment could fail so quickly. 

In June of 2006 Edison informed the NRC that the replacement steam generators to be 
manufactured by Mitsubishi would be fabricated to the same design specifications as the original 
San Onofre Combustion Engineering (CE) steam generators. According to Nuclear Engineering 
International, Edison has admitted that this was a strategic decision to avoid a more thorough 
license amendment and review process. 1 

At SONGS, the major premise of the steam generator replacement project was 
that it would be implemented under the 1OCFR50.59 rule, that is, without prior 
approval by the US Nuclear Regulatory Commission (USNRC). To achieve this 
goal, the RSGs were to be designed as 'in-kind replacement for the OSGs in terms 
of form, fit and function. 2 

Fairewinds finds that there are numerous changes to the San Onofre steam generators that are not 
like-for-like or "in-kind". 

Furthermore, the facts reviewed by Fairewinds makes it clear that if Edison had informed the 
NRC that the new steam generators were not like-for-like, the more thorough NRC licensing 
review process would have likely identified the design problems before the steam generators 
were manufactured. 

Finally, Fairewinds finds that tube plugging is not the solution to the vibration problem 3 and that 
the damaged steam generators will still require major modifications with repair and outage time 
that could last more than 18 months if Edison and Mitsubishi are even able to repair these faulty 
designed steam generators. However, Fairewinds finds that the safest long-term action is the 
replacement of the San Onofre steam generators. 
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Analysis 

The requirements for the process by which nuclear power plant operators and licensees may 
make changes to their facilities and procedures as delineated in the safety analysis report and 
without prior NRC approval are limited by specific regulations detailed in The Nuclear 
Regulatory Commission's 10 CFR Part 50, Domestic Licensing of Production and Utilization 
Facilities, Section 50.59, Changes, Tests and Experiments . 

The implementing procedures for the 10 CFR 50.59 regulations have eight criteria that are 
important for nuclear power plant safety. (These eight criteria are provided in Table 1, footnote 
A below.) 

These implementing procedures created for 10 CFR. 50.59 require that the license be amended 
unless none of these eight criteria are triggered by any change made by Edison at San Onofre. If 
a single criterion is met, then the regulation requires that the licensee pursue a license 
amendment process. 

By claiming that the steam generator replacements were a like-for-like design and fabrication, 
Edison avoided the more rigorous license amendment process. From the evidence reviewed, it 
appears that the NRC accepted Edison's statement and documents without further independent 
analysis. In the analysis detailed below, Fairewinds identified 39 separate safety issues that 
failed to meet the NRC 50.59 criteria. Any one of these 39 separate safety issues should have 
triggered the license amendment review process by which the NRC would have been notified of 
the proposed significant design and fabrication changes. 

As the NRC guidelines state: 

"(c)(1) A licensee may make changes in the facility as described in the final safety 
analysis report (as updated), make changes in the procedures as described in the final 
safety analysis report (as 1.187-A-lupdated), and conduct tests or experiments not 
described in the final safety analysis report (as updated) without obtaining a license 
amendment pursuant to § 50.90 only if: (i)A change to the technical specifications 
incorporated in the license is not required, and (ii) The change, test, or experiment does 
not meet any of the criteria in paragraph (c)(2) of this section." 4 [Emphasis Added] 

In its previous reports, Fairewinds identified at least eight modifications to the original steam 
generators at San Onofre. 

Table 1 below was designed to compare the eight major design modifications that Fairewinds 
identified in its analysis with the eight criteria the NRC applies to the license review process in 
order to determine whether or not a new license amendment process is required. The major 
design changes are located at the top of the table, and the NRC Criteria are listed in the left hand 
column of table. The term SSC stands for Systems, Structures and Components. A green No 
means that the like-for-like criteria were indeed met and that no license amendment was required. 
A red Yes means that Edison should have applied for a license amendment. 

Table 1 shows that 7 out of 8 of the major design changes to the original steam generators meet a 
total of 39 of the NRC's 50.59 criteria requiring amendment to the license. 
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Table 1 

Steam Generator Design Changes Identified By Fairewinds 
Compared With The NRC's Like-For-Like Criteria 



50:59 
Criteria 

(A) 


(B) Remove 
stay 
cylinder 


Change 
tube sheet 


Tube alloy 
change 


Add tubes 


Change 

tube 
support 


Add flow 
restrictor 


Additional 
water 
volume 


Feed water 
distribution 
ring 


i - Accident 
Frequency 
Increase 


Yes(l) 


Yes(l) 


No 


Yes (3,4) 


Yes (3,4,8) 


No 


No 


No 


ii - Increase 

inSSC 
Malfunction 
occurrence 


Yes(l) 


Yes(l) 


No 


Yes (3,4) 


Yes (3,4,8) 


No 


No 


No 


iii - Accident 
consequent 
increase 


Yes(l) 


Yes(l) 


No 


Yes (3,4) 


Yes (3,4,8) 


Yes (2) 


Yes (2,5,6) 


No 


iv - Increase 

inSSC 
consequence 

01 

malfunction 


Yes(l) 


Yes(l) 


No 


Yes (3,4) 


Yes (3,4,8) 


Yes (2) 


Yes (2,5,6) 


No 


v - Create 
unanalysed 
accident 


Yes(l) 


Yes(l) 


No 


No 


No 


Yes (2) 


Yes (2,5,6) 


Yes (3,7,8) 


vi - Create 

new 
malfunction 


Yes (1) 


Yes(l) 


No 


No 


Yes (3,8) 


Yes (2) 


No 


Yes (3,7,8) 


vii - Alter 
fission 
product 
barrier 


Yes(l) 


Yes(l) 


No 


Yes (3) 


No 


No 


No 


No 


viii - Change 
design basis 
evaluation 
method 


Yes (2) 


Yes (2) 


No 


Yes (2) 


Yes (2,8) 


Yes (2) 


Yes (2,5,6) 


No 



Table Footnotes 

A - The criteria listed in the left column in the table above refers to the criteria as laid out in the NRC 
Guidelines 5 which states as follows: 

"(2) A licensee shall obtain a license amendment pursuant to § 50.90 prior to implementing a proposed 
change, test, or experiment if the change, test, or experiment would: 

(i) Result in more than a minimal increase in the frequency of occurrence of an accident previously 
evaluated in the final safety analysis report (as updated); 

(ii) Result in more than a minimal increase in the likelihood of occurrence of a malfunction of a 
structure, system, or component (SSC) important to safety previously evaluated in the final safety 
analysis report (as updated); 

(iii) Result in more than a minimal increase in the consequences of an accident previously evaluated in 
the final safety analysis report (as updated); 

(iv) Result in more than a minimal increase in the consequences of a malfunction of an SSC important 
to safety previously evaluated in the final safety analysis report (as updated); 

(v) Create a possibility for an accident of a different type than any previously evaluated in the final 
safety analysis report (as updated); 

(vi) Create a possibility for a malfunction of an SSC important to safety with a different result than 
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any previously evaluated in the final safety analysis report (as updated); 

(vii) Result in a design basis limit for a fission product barrier as described in the FSAR (as updated) 

being exceeded or altered; or 

(viii) Result in a departure from a method of evaluation described in the FSAR (as updated) used in 
establishing the design bases or in the safety analyses." 

B - The horizontal axis contains a list of design changes made by Edison and whether they meet or have 
not met the criteria as set out in 10 CFR 50.59. 

1 - The Steam Generator Replacement Project modified the tube sheets and stay cylinder that are a 
containment barrier - The NRC was not informed nor did it specifically approve these changes to 
the containment barrier as they were apparently not addressed under Edison's analysis for the 10 
CFR 50.59 process; 

2 - The Mitsubishi thermo hydraulic code is inadequate to assess flow inside the Steam 
Generators that dramatically affect the ability to cool the nuclear reactor core in the event of an 
accident; 

3 - The Steam Generator Replacement Project increases the consequences of a steam line break 
accident; 

4 - The Steam Generator Replacement Project has already proven to increase the frequency of 
tube failure; 

5 - The Steam Generator Replacement Project changed the volume of primary coolant because 
more tubes were added, which changes the Final Safety Analysis Report; 

6 - The Steam Generator Replacement Project changed the flow rate of primary coolant, which 
changes the Final Safety Analysis Report; 

7 - The Steam Generator Replacement Project changed the potential for water hammer. Given 
that the Mitsubishi thermo hydraulic code is inadequate, the potential for water hammer is 
increased; 

8 - The Steam Generator Replacement Project created steam binding at top of steam generator. 
The steam generator is designed to remove heat in the event of an accident and its role has been 
compromised. 



Ramifications Of Edison's Decision To Avoid The License Amendment Process 

Edison's strategic goal was to avoid the process of license amendment according to the January 
2012 article in Nuclear Engineering International NEI Magazine 6 . Had Edison notified the NRC 
that the new steam generators at San Onofre were not a like-for-like replacement, a more 
thorough review through the license amendment process would have been required. Given that 
scenario, it is likely that the requisite and thorough NRC review would have identified the design 
and fabrication inadequacies that appear to have caused the San Onofre steam generator tube 
failure. 

More specifically, Fairewinds believes that the NRC would have identified the inadequacy of the 
Mitsubishi Heavy Industry computer code applied to validate the tube design and vibration 
pattern prior to fabrication. Mitsubishi's computer code was simply not capable of analyzing 
Combustion Engineering (CE) designs like San Onofre and was only qualified for Westinghouse 
designs that are not similar to the original CE steam generator design. In NRC licensing jargon, 
the Mitsubishi design codes were not benchmarked for the CE Design 7 . 

While Mitsubishi Heavy Industry has been supplying steam generators for many years in Japan, 
it did so under a specific license from Westinghouse for Westinghouse nuclear reactors. 
Although Mitsubishi made several incremental changes to the Westinghouse design, such as 
switching to alloy 690 tubing and the use of stainless steel broached plate tube supports, 
Mitsubishi has had very little experience with the tight tube pitch and the egg crate design used 
in the original CE design for San Onofre. 
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Figure 1: Broached Tube Support Plate - Designed To Keep Tubes From Rattling 







Broached (quatrefoil and trefoil) tube support plates (TSPs) 8 
ure 2: Eggcrate Tube Support Plate - Designed To Keep Tubes From Rattling 
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Horizontal Tube Supports (Eggcrate) 9 
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The original steam generators designed and manufactured by CE for San Onofre were 
successfully operated 28 years. Moreover the original steam generators had a triangular tube 
pitch pattern, very closely packed U-tubes, and unique egg-crate tube supports that kept the tubes 
from vibrating and colliding. The pitch to diameter ratio of tubes in the original CE generators is 
dramatically different from any of the Westinghouse generators fabricated by Mitsubishi. 
Moreover, an NRC licensing review would have identified the fact that the Mitsubishi computer 
design code, which is based upon Westinghouse models, was not appropriate for design changes 
to the San Onofre replacement steam generators originally designed by CE. 

Another problem with the San Onofre steam generators is that Edison and Mitsubishi made a 
very significant design change that magnified the San Onofre steam generator stresses and 
vibrations by removing the main structural pillar called the stay cylinder in order to fit an 
additional 400 tubes into the unique and already tightly packed design. Furthermore, this design 
is also bigger than anything Mitsubishi Heavy Industries (MHI) had ever fabricated or designed. 
The NRC license amendment review process would likely have identified these and other 
problems. 



The Actual Steam Generator Problem 

As water moves vertically up in a steam generator, the water content reduces as more steam is 
created. When the volume of steam is much greater than water then the flow resistance of the 
water/steam mixture passing through the tube supports accounts for one third of the total 
resistance at the top of the steam generator. Therefore to avoid vibration at the top of the tubes, 
Mitsubishi needed to specifically analyze the type of tube support to use in this unique 
application. 

The flow resistance of the Mitsubishi broached plate is much higher than that of the original 
Combustion Engineering egg crate design because the tubes are so tightly packed in the original 
CE San Onofre steam generators. By reviewing the documents thus far produced, it appears that 
due to Mitsubishi's fabrication experience with broached plates, both Edison and Mitsubishi 
missed this key difference in the design and fabrication of the new San Onofre steam generators. 

Not only is Mitsubishi unfamiliar with the tightly packed CE design, but also Edison's engineers 
created so many untested variables to the new fabrication that this new design had a significantly 
increased risk of failure. As a result of the very tight pitch to diameter ratios used in the original 
CE steam generators, Mitsubishi fabricated a broached plate design that allows almost no water 
to reach the top of the steam generator. 

The maximum quality of the water/steam mixture at the top of the steam generator in the U-Bend 
region should be approximately 40 to 50 percent, i.e. half water and half steam. With the 
Mitsubishi design the top of the U-tubes are almost dry in some regions. 10 Without liquid in the 
mixture, there is no damping 11 against vibration, and therefore a severe fluid-elastic instability 
developed. 
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In response to the Edison/Mitsubishi steam generator changes, the top of the new steam 
generator is starved for water therefore making tube vibration inevitable. Furthermore, the 
problem appears to be exacerbated by Mitsubishi's three-dimensional thermal-hydraulic analysis 
determining how the steam and water mix at the top of the tubes that has been benchmarked 
against the Westinghouse but not the Combustion Engineering design. 

The real problem in the replacement steam generators at San Onofre is that too much steam and 
too little water is causing the tubes to vibrate violently in the U-bend region. The tubes are 
quickly wearing themselves thin enough to completely fail pressure tests. Even if the new tubes 
are actively not leaking or have not ruptured, the tubes in the Mitsubishi fabrication are at risk of 
bursting in a main steam line accident scenario and spewing radiation into the air. 



This Tube Damage Cannot Be Repaired 

Edison claims that the proximate cause of these U-tube failures at San Onofre is high vibration, 
and it has embarked upon a process of plugging some of these damaged tubes in hopes of 
quickly restarting one or both units. Fairewinds believes that this damage is occurring on the 
outside of the tubes where they collide with each other, while access to the tubes for repair 
and/or plugging can only be conducted from inside the tubes. Space limitations due to the tight 
fit of the 9,700 tubes (19,400 holes in the tube sheet) in each steam generator have made it 
impossible to access the outside of the U-tubes for inspection where the wear is actually 
occurring. 

Presently, the Edison approach is to plug tubes in the most heavily damaged zone of each steam 
generator. Plugging the tubes only eliminates the radioactive water inside the tubes, but it does 
not eliminate the vibration, so the plugged tubes will continue to vibrate and damage adjacent 
tubes. More than 500 tubes have already been plugged in Unit 2 and more than 800 tubes have 
been plugged in Unit 3. 12 The number of plugged tubes is still considerably smaller than the 
number of tubes already ascertained as damaged in both steam generators. To date, Edison has 
not provided adequate data to compare damaged tubes to plugged tubes. 

Initially, in March 2012, Edison claimed that as part of the Electric Power Research Institute's 
(EPRI) criteria used in the in-situ pressure testing of the Steam Generators, it was required to 
plug about one dozen tubes in the San Onofre steam generators. However, in May 2012, Edison 
announced it had plugged 1300 tubes, more than one hundred times the number of tubes required 
by the EPRI criteria. According to the industry steam generator experts interviewed by 
Fairewinds, Edison did not plug these additional tubes because they had failed, but rather Edison 
needed to plug these particular tubes because they would likely fail during a main steam line 
break accident. 

If a steam line break accident were to occur, the depressurization of the steam generator caused 
by the steam line break coupled with the lack of water at the top of the steam generators would 
cause cascading tube failures, involving hundreds of tubes. The cascading tube failures would 
pop like popcorn and the cascading failures would cause excessive offsite radiation exposures. 
In an attempt to avoid a severe steam line break accident Edison prophylactically plugged 
additional tubes. 
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Fairewinds investigation has found that plugging the tubes is not a sure solution, because it 
fails to deal with the root causes of a failed design and it relies upon the incorrectly applied 
Mitsubishi 3 -Dimensional steam analysis to determine which tubes should be plugged. 
Realistically, the 3-D steam analysis is not accurate enough to apply to such important safety- 
related determinations. To make such mathematical risk 3-D analysis, a very large margin of 
error must be applied, and that has not been done. For example, if the 3-D steam analysis 
determines that plugging 100 tubes is a solution, then plugging ten times that number might be 
the appropriate solution due to the mathematical errors in the 3-D analysis being applied by 
Edison and Mitsubishi. 

Fairewinds concludes that plugging the tubes will never solve the underlying problem because 
vibration is the result not the root cause of the steam generator problems at San Onofre. The 
actual problem is a variety of design changes that have caused too much steam and too little 
water at the top of the steam generators. Plugging tubes cannot repair these design 
changes created and that are causing the tubes to collide with each other. 

The tubes that Edison has already plugged on the inside will continue to vibrate because they are 
being pushed by steam and water from the outside. Therefore Fairewinds concludes that 
Edison's solution of plugging the inside of the tubes will not lessen the risk of an accident or stop 
the ongoing vibrational damage that is occurring to the inaccessible outside of the San Onofre 
steam generator tubes. 



Options For Continued Operation Of The San Onofre Reactors 

Complete Replacement 

The ongoing plugging of the tubes will not eliminate the vibrational failure mechanism 
causing tube failures. Over time, the damaged tubes that are plugged will in turn damage 
more tubes. Therefore, Fairewinds believes that the only sure solution to this significant 
safety issue is to once again cut open the reactor containment and install new steam 
generators that replicate the original CE design. 

Due to the significant risk of a steam generator tube rupture accident in such a highly 
populated and vulnerable area, both San Onofre Unit 2 and Unit 3 should remain shut down 
until such a significant safety threat can be mitigated with the fabrication of new like-for- 
like steam generators adhering to the original CE design. If all the appropriate steps are 
taken in design and fabrication of new CE replica steam generators, and the proper 
procedures are taken to repair and reseal the San Onofre containment coupled with 
requisite NRC oversight, Fairewinds estimates that the entire process might take Edison 
approximately four years and cost in excess of $800,000,000, not including replacement 
power while the Units remain shut down. 
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Repair In Place 

While technically this would be an extremely challenging repair process, it may be possible 
to cut the steam generators apart while still inside the containment. Such a process would 
take approximately 18 months to make repairs and then weld the steam generators back 
together again without cutting the containment open. Cutting the top off the steam 
generators would allow construction personnel access so that additional supports could be 
inserted into the U-tube region. Smaller replacement packages would fit through the 
existing equipment hatch and the containment would not be compromised another time. 
The cost for these repairs would be less than completely redesigning and manufacturing 
new steam generators and replacement power costs would be less. However, it is still 
reasonable to estimate that this cost would exceed $400,000,000 without replacement 
power, not including replacement power while the Units remain shut down. 

There are two possible alternatives, both of which would require an additional analysis of 
the overall steam generator flow patterns to ensure that no new problems are created in 
the process of attempting to mitigate the damage from these design flaws and fabrication 
errors. The two alternatives are: 

1. Because too much steam and too little water in the U-bend region cause the 
vibration problems, it might be possible to reduce the steam/water mixture 
qualities in the U-bend area by changing the internal structures to divert some of the 
internal recirculating flow into the U-bend region. 

2. Another possible solution would require replacing the steam-water separators. The 
Mitsubishi separators require a water level that is quite low in the steam drum, and 
cannot be raised. Changing the separators to a different design may allow more 
water to reach the top of the tubes and thereby stop the tube vibration and wear. 

Power Reduction 

Reducing power does not provide a remedy for the underlying structural problems that are 
creating the vibration that has damaged and will continue to damage tubes deep inside the 
San Onofre steam generator. Edison has suggested that plugging tubes and operating at 
indeterminate reduced power levels for the remainder of the life of the plant may be a 
solution to the San Onofre tube vibration problem. Unfortunately this course of action 
would leave San Onofre operating with a significant safety risk if the NRC were to allow the 
reactors to restart. 

The concept of reducing the power output from the San Onofre reactors will not change 
either the inside steam generator tube water temperature or the steam temperatures 
outside of the tubes. Reducing the power output will also not change the 2200-pound per 
square inch pressure within the tubes or the 1,000-pound pressure outside the tubes. 
Operating at reduced power will not prevent previously damaged tube supports and 
plugged tubes from vibrating and damaging surrounding tubes and tube supports, and it 
will worsen the existing damage. 

More importantly, Fairewinds concern is that operating the San Onofre reactors at a lower 
power and flow rate might actually create a resonate frequency within the steam 
generators at which some of the tubes will vibrate as bad or worse than they did originally. 



11 of 13 



Because the plugged tubes are now filled with air their weight has changed, and therefore 
the plugged tubes will vibrate with a different amplitude and frequency. The inaccuracies 
in the Edison and Mitsubishi computer code do not allow Edison and Mitsubishi to conduct 
a resonant frequency analysis proving that such a problem will not occur. 

It is impossible to determine exactly what is happening inside an operating steam 
generator. For example, at Millstone 2, a smaller CE reactor, the steam generator tube 
supports began to disintegrate due to vibration, and there was no method to alert the 
operations staff that such deterioration was occurring. This challenging problem was 
finally detected when the Millstone 2 was shut down for a refueling, and small cameras 
meant to inspect the steam generator found rubble on the tube sheet at the base of the 
tubes. 

Historical evidence from other operating nuclear reactors that have attempted to mitigate 
vibrational damage by using power reductions rather than solving the resonant frequency 
issues have in fact compromised other nuclear safety related components by operating at 
reduced power. 

• In 2002 the Exelon Quad Cities Nuclear Power Plant in Illinois operated its Unit 2 
reactor at reduced power in order to eliminate vibrationally induced damage 
causing high moisture carryover in its steam dryer. While the power reduction 
temporarily reduced moisture carryover, the problem reoccurred and a shutdown 
was ordered causing an extended unplanned outage. Vibrationally induced severe 
cracking was discovered in the steam dryer and repaired. Following an analysis and 
subsequent repairs, Exelon claimed to have rectified the Quad Cities Unit 2 
problems only to be forced in 2003 to once again attempt operation at a reduced 
power level when vibrationally induced steam dryer moisture carryover became 
excessive. Following this second attempt to operate the reactor at a reduced power 
level, pieces of the dryer as large as a man broke off and damaged nuclear power 
safety related components, and a second unplanned extended outage ensued. Once 
again, vibration was determined to be the cause of the gross failure and another 
unplanned and forced outage. Finally, following years of analysis and two damaged 
steam dryers, Quad Cities made major piping modifications that are alleged to have 
eliminated harmonic frequencies, prevented further component damage, and 
allowed Unit 2 to eventually return to full power production. 13 

• A second example of a failed attempt to reduce power to solve vibrationally induced 
resonance frequency problems occurred at the Susquehanna nuclear plant in 
Pennsylvania. During the mid 1990s, a vibrationally induced failure in the jet pump 
sensing lines occurred at Susquehanna. This failure was attributed to the vane 
passing frequency from the recirculation pumps causing harmonic vibration of the 
lines. Like Quad Cities, Susquehanna attempted to implement a power reduction in 
order to minimize the harmonic vibrations. Unfortunately, the resonant vibration 
issues continued to damage systems after the power was reduced thereby forcing an 
unplanned outage and extensive modifications and repairs. 14 
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Conclusion 

In conclusion, the NRC has stated that nuclear power plants like San Onofre cannot risk 
compromising critical safety systems and possible radiological contamination in an effort 
to return to operation before a thorough root cause analysis, modifications, and subsequent 
repairs are adequately reviewed by the NRC and implemented. Historical evidence has 
proven that power reductions do not solve underlying and serious degradation problems, 
resonance frequency issues. Rather, power reductions can significantly increase the risk of 
unplanned, forced outages during times of peak demand and can cause significant risk to 
public health in the event of a single tube rupture or a series of ruptures if the main steam 
line were to break. 

Finally, if a steam-line accident were to occur, vibrationally induced tube damage at San 
Onofre could cause an inordinate amount of radioactivity to be released outside of the 
containment system compromising public health and safety in one of the most heavily 
populated areas in the entire United States. 

Note : 

This report represents the opinion of Fairewinds. Industry insiders, who have had lengthy 
careers in steam generator design, fabrication, and operation, and who have chosen to remain 
anonymous, have assisted Fairewinds with research for this report, but are not responsible for its 
content. 
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